An Efficient Method for online Detection of Polychronous Patterns in Spiking Neural Network

Polychronous neural groups are effective structures for the recognition of precise spike-timing patterns but the detection method is an inefficient multi-stage brute force process that works off-line on pre-recorded simulation data. This work presents a new model of polychronous patterns that can capture precise sequences of spikes directly in the neural simulation. In this scheme, each neuron is assigned a randomized code that is used to tag the post-synaptic neurons whenever a spike is transmitted. This creates a polychronous code that preserves the order of pre-synaptic activity and can be registered in a hash table when the post-synaptic neuron spikes. A polychronous code is a sub-component of a polychronous group that will occur, along with others, when the group is active. We demonstrate the representational and pattern recognition ability of polychronous codes on a direction selective visual task involving moving bars that is typical of a computation performed by simple cells in the cortex. The computational efficiency of the proposed algorithm far exceeds existing polychronous group detection methods and is well suited for online detection.Comment: 17 pages, 8 figure

Multi-objective Search of Robust Neural Architectures against Multiple Types of Adversarial Attacks

Many existing deep learning models are vulnerable to adversarial examples that are imperceptible to humans. To address this issue, various methods have been proposed to design network architectures that are robust to one particular type of adversarial attacks. It is practically impossible, however, to predict beforehand which type of attacks a machine learn model may suffer from. To address this challenge, we propose to search for deep neural architectures that are robust to five types of well-known adversarial attacks using a multi-objective evolutionary algorithm. To reduce the computational cost, a normalized error rate of a randomly chosen attack is calculated as the robustness for each newly generated neural architecture at each generation. All non-dominated network architectures obtained by the proposed method are then fully trained against randomly chosen adversarial attacks and tested on two widely used datasets. Our experimental results demonstrate the superiority of optimized neural architectures found by the proposed approach over state-of-the-art networks that are widely used in the literature in terms of the classification accuracy under different adversarial attacks

Evolutionary Multiobjective Optimization Driven by Generative Adversarial Networks (GANs)

Recently, increasing works have proposed to drive evolutionary algorithms using machine learning models. Usually, the performance of such model based evolutionary algorithms is highly dependent on the training qualities of the adopted models. Since it usually requires a certain amount of data (i.e. the candidate solutions generated by the algorithms) for model training, the performance deteriorates rapidly with the increase of the problem scales, due to the curse of dimensionality. To address this issue, we propose a multi-objective evolutionary algorithm driven by the generative adversarial networks (GANs). At each generation of the proposed algorithm, the parent solutions are first classified into real and fake samples to train the GANs; then the offspring solutions are sampled by the trained GANs. Thanks to the powerful generative ability of the GANs, our proposed algorithm is capable of generating promising offspring solutions in high-dimensional decision space with limited training data. The proposed algorithm is tested on 10 benchmark problems with up to 200 decision variables. Experimental results on these test problems demonstrate the effectiveness of the proposed algorithm

Evolving hierarchical gene regulatory networks for morphogenetic pattern formation of swarm robots

Morphogenesis, the biological developmental process of multicellular organisms, is a robust self-organising mechanism for pattern formation governed by gene regulatory networks (GRNs). Recent findings suggest that GRNs often show the use of frequently recurring patterns termed network motifs. Inspired by these biological studies, this paper proposes a morphogenetic approach to pattern formation for swarm robots to entrap targets based on an evolving hierarchical gene regulatory network (EH-GRN). The proposed EH-GRN consists of two layers: The upper layer is for adaptive pattern generation where the GRN model is evolved by basic network motifs, and the lower layer is responsible for driving robots to the target pattern generated by the upper layer. Obstacle information is introduced as one of environmental inputs along with that of targets in order to generate patterns adaptive to unknown environmental changes. Besides, splitting or merging of multiple patterns resulting from target movement is addressed by the inherent feature of the upper layer and the k-means clustering algorithm. Numerical simulations have been performed for scenarios containing static/moving targets and obstacles to validate the effectiveness and benefit of the proposed approach for complex shape generation in dynamic environments

A Survey on Surrogate-assisted Efficient Neural Architecture Search

Neural architecture search (NAS) has become increasingly popular in the deep learning community recently, mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks (DNNs). However, NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS, and training DNNs is computationally intensive. To solve the major limitation of NAS, improving the efficiency of NAS is essential in the design of NAS. This paper begins with a brief introduction to the general framework of NAS. Then, the methods for evaluating network candidates under the proxy metrics are systematically discussed. This is followed by a description of surrogate-assisted NAS, which is divided into three different categories, namely Bayesian optimization for NAS, surrogate-assisted evolutionary algorithms for NAS, and MOP for NAS. Finally, remaining challenges and open research questions are discussed, and promising research topics are suggested in this emerging field.Comment: 18 pages, 7 figure

CES-485 Approximating the Set of Pareto Optimal Solutions in Both the Decision and Objective Spaces by an Estimation of Distribution Algorithm

Most existing multiobjective evolutionary algorithms aim at approximating the PF, the distribution of the Pareto optimal solutions in the objective space. In many real-life applications, however, a good approximation to the PS, the distribution of the Pareto optimal solutions in the decision space, is also required by a decision maker. This paper considers a class of MOPs, in which the dimensionalities of the PS and PF are different so that a good approximation to the PF might not approximate the PS very well. It proposes a probabilistic model based multiobjective evolutionary algorithm, called MMEA, for approximating the PS and the PF simultaneously for a MOP in this class. In the modelling phase of MMEA, the population is clustered into a number of subpopulations based on their distribution in the objective space, the PCA technique is used to detect the dimensionality of the centroid of each subpopulation, and then a probabilistic model is built for modelling the distribution of the Pareto optimal solutions in the decision space. Such modelling procedure could promote the population diversity in both the decision and objective spaces. To ease the burden of setting the number of subpopulations, a dynamic strategy for periodically adjusting it has been adopted in MMEA. The experimental comparison between MMEA and the two other methods, KP1 and Omni-Optimizer on a set of test instances, some of which are proposed in this paper, have been made in this paper. It is clear from the experiments that MMEA has a big advantage over the two other methods in approximating both the PS and the PF of a MOP when the PS is a nonlinear manifold, although it might not be able to perform significantly better in the case when the PS is a linear manifold